Assessing Student Learning Through Keyword Density Analysis of Online Class Messages

This paper presents a novel approach that automatically assesses student class performance by analyzing keyword density in online class messages. Class performance of students is evaluated from three perspectives: the quality of their course work, the quantity of their efforts, and the activeness of their participation; three measures keyword density, message length, and message count, are derived from class messages respectively. Noun phrases are extracted from the messages and weighted based on frequency and length. Keyword density is defined as the sum of weights of noun phrases appearing in a student’s messages. Message length and message count are the number of words and the number of messages in the student’s messages respectively. The three measures are combined into a linear model, in which each measure accounts for a certain proportion of a final score, called performance indicator. The experiment shows that there is a high correlation between the performance indicator scores and the actual grades assigned by instructors. The rank orders of students by the performance indicator score are highly correlated with those by the actual grades as well. Evidences of the supplemental role of the computer assigned grades are also found.